Skip to content

test(metadata-protocol,objectql): key the protocol stub engines by table - #16651

Merged
os-musk merged 3 commits into
mainfrom
claude/issue-16225-protocol-harness-table-keyed-stub
Sep 7, 2026
Merged

test(metadata-protocol,objectql): key the protocol stub engines by table#16651
os-musk merged 3 commits into
mainfrom
claude/issue-16225-protocol-harness-table-keyed-stub

Conversation

@os-musk

@os-musk os-musk commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Fixes #16225

Eleven protocol test harnesses gave their stub engine one flat row map and told the tables apart in exactly one place — the insert early-return for sys_metadata_audit. find / findOne ignored the table argument entirely, so every other table a save writes (sys_metadata_history, sys_metadata_commit) landed in the map that answered reads of sys_metadata.

Each stub now holds a map from table NAME to that table's rows, reached through a tableOf(name) accessor — the shape protocol.runtime-gate-stored-universe.test.ts (#16223) established, copied rather than reinvented. rows stays bound to sys_metadata, so every existing assertion reads the table it names.

Why the remaining seven were worth touching

"Had the assertion been written the other way round — or had the harness been used to prove a store-reading change works — it would have passed for a reason that exists nowhere but in the stub."

The defect was caught by the polarity of one assertion, not by anything structural. That is the whole argument: this shape does not produce a red someone investigates, it produces a green that closes a card.

The incident, reproduced and pinned

Driving one DRAFT saveMetaItem through a converted harness and dumping each table:

TABLE sys_metadata         rows=1  state="draft"
TABLE sys_metadata_history rows=1  keys=[id,event_seq,type,name,version,operation_type,metadata,
                                         checksum,previous_checksum,change_note,source,
                                         organization_id,recorded_by,recorded_at]   no `state` key
TABLE sys_metadata_audit   rows=1  keys=[id,occurred_at,actor,source,type,name,organization_id,
                                         operation,outcome,code,lock_state,lock_overridden,
                                         request_id,note]                            no `state` key

All three are addressed to the same (type, name, organization_id), and the two journal rows carry no state. A flat map hands them back to a sys_metadata read; a harness that also models sys_metadata.state's declared defaultValue: 'active' — correctly, which is what made it convincing — serves a DRAFT-only artifact back as an ACTIVE metadata row.

Two pins, one per package, assert that a sys_metadata read answers with the store row and nothing else:

  • packages/metadata-protocol/src/protocol.dashboard-dataset-publish-gate.test.ts
  • packages/objectql/src/protocol-save-meta-repo-path.test.ts

Each is guarded by a firing control (the save must really have written the journals, or the read below measures an empty universe) and asserts on store-only columns (state, checksum) rather than on a row count. That last choice is load-bearing: a journal row is addressed to the same key keyOf builds, so under a merged map it can simply OVERWRITE the store row and a count- or name-based assertion still reads one row of the right name. The first draft of the objectql pin was vacuous for exactly that reason and was rewritten.

Ablation — one leg per package

Mutation: tableOf re-merged to hand every caller the same map. On-disk proof taken both ways (deleted text 1 -> 0, injected text present), blob hash compared against HEAD:path before and after.

leg mutated restored
protocol.dashboard-dataset-publish-gate.test.ts 1 failed, 6 passed — only the #16225 pin blob bba0bd3e = HEAD, git diff HEAD empty, git status --porcelain empty, 7 passed
protocol-save-meta-repo-path.test.ts 5 failed, 9 passed — the #16225 pin plus 4 checksum/seq assertions blob 00f32020 = HEAD, git diff HEAD empty, git status --porcelain empty, 14 passed

The metadata-protocol leg is the sharper reading: exactly one test in that file distinguishes the two shapes, and it is the one this PR adds. The four extra reds on the objectql leg come from the audit row colliding with the store row under that file's keyOf once the audit_skip early-return is gone as well — the ablation removes the whole change, not just half of it.

Scope rule 2 — the shape population, measured

A population defined by a spelling is not a population defined by the shape, so the shape was measured directly. Both halves of the card's shape are read from each fake engine's OWN parameter name, so the expression is spelling-independent. Two references deliberately do not count as using the table argument: assertEngineFindOnePredicate(p, opts) (checks the caller's predicate, never selects a row — this is why the eight read as table-aware to a naive grep) and the sys_metadata_audit early-return itself.

Script: shape-census.mjs, run over packages/metadata-protocol packages/objectql, 93 files declaring an async insert.

axis before after
insert puts every table in one container (FLAT-WRITE) 15 4
that and find/findOne answer without asking which table (the full incident shape) 9 1
the card's exact grep, if (_t === 'sys_metadata_audit') return 8 0

Controls, both directions, from the same run: the reference fix protocol.runtime-gate-stored-universe.test.ts, protocol.package-publish-audit-rows.test.ts and sys-metadata-repository.contract.test.ts are all table-keyed and are all absent from the before-listing; all 8 files the card names are present in it. A first version of the census reported 120 files and a second reported the reference fix itself as blind — both were parser bugs (the type-literal brace in Promise object return types read as the method body), found by those controls.

Triage of the residue:

  • The 9th full-shape hit is packages/objectql/src/hook-run-as.test.ts — a recording engine (insert pushes to a seen log and returns a constant; there is no row map, and its parameter is _o, an object not a table). Not this family, not converted.
  • Of the 15 FLAT-WRITE files, 4 persist nothing on insert (protocol.delete-object-registry-unregister, protocol.delete-receipt-wording, protocol.driver-text-disclosure, plus hook-run-as); their rows are pre-seeded and their findOne already guards table !== 'sys_metadata'. They are the 4 that remain, and they are false positives of the axis, not residue.
  • The other 3 beyond the card's 8 — protocol.container-issue-descent, protocol.invalid-metadata-422-face-inventory, protocol.save-union-issues — carry the flat map but answer null / [] from every read, so no row of one table can be served as a row of another. What the flat map still did there was make rows.size the total of every table one save writes. Same mechanical change, so converted here; the sub-shape is stated in each file's own header note.

So the spelling did not under-count the incident shape — it coincides with it at 8 — but it did under-count the flat-map family by 3.

Scope rule 1 — reds

None. All 11 harnesses were green before conversion (162 tests) and green after; the full suites of both packages are green (2433 + 4802). Nothing was passing because a sys_metadata_history row answered a sys_metadata read, which matches the card's own statement that the flaw is latent. Nothing was re-merged, nothing was relaxed, no card was filed for a red because there was none.

The audit_skip early-return is gone

With the tables separated it has no job left — audit rows land in sys_metadata_audit, which no read in these files consults. Removing it also clears the trap protocol.package-publish-audit-rows.test.ts names in its own header: a suite that later wants to assert on audit rows written in one of these files would otherwise be born vacuous. No test asserts on a row id, so the nextId shift it causes is unobservable (verified: zero matches for r_[0-9] in the eight).

Clause-② — re-derived from the diff that exists

No. The diff is 11 *.test.ts files in two packages and nothing else. Nothing exported moves, no accept set widens or narrows, no published payload gains a key, no product code is touched. needs:contract-review not hung. No product change was needed to make any converted harness honest.

Changeset: test-only, nothing publishes ⇒ skip-changeset.

Verification

  • Gate derivation, node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands with no hand-listed paths, reconciled with --ran:
    Run reconciliation — 48 derived, 46 run, 2 NOT-MEASURED, 0 UNRUN.
  • The 2 NOT-MEASURED, in the gates' own words, both refusing on an unbuilt dependency closure and neither chaseable with a bigger heap:
    • pnpm check:dual-build-cjs-loads — "PREREQUISITE NOT MET — this gate reads built output, and some package has no dist/. Run pnpm build first. ⛔ This is NOT a pass: nothing was measured."
    • pnpm check:type-check-debt — "check-type-check-coverage: PREREQUISITE NOT MET … --re-measure cannot run: 17 workspace dependencies of the ledgered packages have no built type entry point on disk." Its own report names this process's limit as 8240 MB with no OOM.
  • The two live risks this diff carries are both green: pnpm check:engine-double-contract exit 0, pnpm check:test-source-alias exit 0. No gate asked for a ledger row, so no ledger was touched.
  • pnpm --filter @objectstack/metadata-protocol test — 169 files, 2433 passed, 10 skipped.
  • pnpm --filter @objectstack/objectql test — 284 files, 4802 passed.
  • pnpm --filter @objectstack/metadata-protocol --filter @objectstack/objectql typecheck — exit 0, and tsc --noEmit --listFiles confirms the converted test files are inside the typecheck program (not excluded).
  • eslint . --no-inline-config over the whole repo at 220bfdba: 6301 files, 0 errors, 0 warnings, exit 0. No narrowing claimed — this is the whole population.
  • Every exit code above was captured after a redirect, never through a pipe.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg


Generated by Claude Code

Eleven protocol harnesses gave their stub engine one flat row map and told
the tables apart in exactly one place — the `insert` early-return for
`sys_metadata_audit`. `find`/`findOne` ignored the table argument entirely,
so every other table one save writes (`sys_metadata_history`,
`sys_metadata_commit`) landed in the map that answered reads of
`sys_metadata`.

Each stub now holds a map from table NAME to that table's rows, reached
through a `tableOf(name)` accessor, copying the shape
`protocol.runtime-gate-stored-universe.test.ts` established. `rows` stays
bound to `sys_metadata`, so existing assertions read the table they name.
The `audit_skip` early-return goes with it: with the tables separated it has
no job left, and it was a standing trap for any future assertion about audit
rows written in these files.

Two pins make the change a measurement rather than a rename — one per
package. A DRAFT save writes `sys_metadata`, `sys_metadata_history` and
`sys_metadata_audit`, all three addressed to the same `(type, name)`, and
the journal rows carry no `state`; each pin asserts a `sys_metadata` read
answers with the store row and nothing else, behind a firing control that
the journals really were written.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg
@github-actions github-actions Bot added the size/l label Sep 7, 2026
@os-musk os-musk added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 7, 2026 — with Claude
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 4fe00b80e7d2da485cae7a777b29609770d8c80bpackageMentionDocs.

@github-actions github-actions Bot added the tests label Sep 7, 2026
`tsc --noEmit` refused the metadata-protocol pin with TS2352: `Row` declares
no index signature, so asserting `Row[]` to `Record<string, unknown>[]` is not
a legal widening. `Partial<Row>` is legal AND true of these rows — a journal
row carries `type` and `name` and carries no `state`, which is exactly what
the assertions beside it read. The objectql pin's `any[]` is spelled the same
way for the same reason, replacing a cast that typechecked while saying
nothing.

Vitest never type-checks, so the suites were green over code tsc refuses; the
error was only ever reachable through the package's own `typecheck` script,
which this branch had last run before either pin existed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg

os-musk commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

Patch round — Type Check · workspace red, reproduced and fixed at ce0aac91

Reproduced locally, one error, and it is mine:

> @objectstack/metadata-protocol@17.3.0 typecheck
> tsc --noEmit

src/protocol.dashboard-dataset-publish-gate.test.ts(397,25): error TS2352: Conversion of type
'Row[]' to type 'Record<string, unknown>[]' may be a mistake because neither type sufficiently
overlaps with the other. If this was intentional, convert the expression to 'unknown' first.
  Type 'Row' is not comparable to type 'Record<string, unknown>'.
    Index signature for type 'string' is missing in type 'Row'.
Exit status 2

Row in that harness declares seven named fields and no index signature, so Row[] as Record<string, unknown>[] is not a legal widening. The line is in the #16225 pin this PR adds — not in any converted harness.

Why the local run said 0 and CI said 2

Not an environment difference, and none of the three candidate explanations. packages/metadata-protocol's typecheck is a single leg, tsc --noEmit, at both my base and the merged origin/main — the three-leg script (tsc --noEmit && tsc --noEmit -p tsconfig.scripts.json && pnpm check:test-typecheck) is objectql's. No ledger was involved: objectql's check:test-typecheck reports 44 file(s) / 242 error(s) / 69 pinned signature(s) held, byte-identical to the reading before this branch, so nothing moved and no ledger row was written.

The real cause is a stale measurement. The ordering was: convert the 8 → run typecheck (exit 0) → convert 3 more → write both pins → rewrite both pins → commit → merge origin/main. The typecheck ran once, before either pin existed, and was never re-run. Vitest never type-checks, so the suites stayed green over code tsc refuses, and the whole-workspace Type Check job was the first thing to look.

The report's two claims were inconsistent on their face and that was the tell I missed: --listFiles proved the converted .test.ts files are inside the typecheck program, which is exactly why tsc can see the pin — so an exit 0 could only have come from a tree that did not contain it.

The fix

Partial<Row>[] in both pins. It is legal, and it is the honest type: a journal row carries type and name and carries no state — which is precisely what the assertions beside it read. The objectql pin's as any[] is respelled the same way, replacing a cast that typechecked while asserting nothing. No product code, no assertion relaxed, no map re-merged; the diff is still 11 test files.

Re-verified in full at ce0aac91

  • pnpm --filter @objectstack/metadata-protocol typechecktsc --noEmit, exit 0.
  • pnpm --filter @objectstack/objectql typecheck — all three legs, exit 0, ending check:test-typecheck: OK — @objectstack/objectql's test layer compiles under packages/objectql/tsconfig.test.json; 44 file(s) / 242 error(s) / 69 pinned signature(s) held in test-typecheck-debt.json.
  • Both run with the dependency closure built (pnpm --filter '<pkg>^...' build), the same world turbo gives the CI job.
  • pnpm --filter @objectstack/metadata-protocol test — 169 files, 2433 passed, 10 skipped. pnpm --filter @objectstack/objectql test — 285 files, 4807 passed.
  • Gates re-derived at ce0aac91 (identical 48-command list) and re-run: Run reconciliation — 48 derived, 46 run, 2 NOT-MEASURED, 0 UNRUN. Same two prerequisite refusals; check:type-check-debt still exits 3 with this card's two closures built, which confirms it is the ledgered packages' closure and not heap.
  • eslint . --no-inline-config at ce0aac91 — 6301 files, 0 errors, 0 warnings, exit 0.
  • Ablation re-run on the new head, both legs: metadata-protocol 1 failed / 6 passed (only the pin), objectql 5 failed / 9 passed; restored blobs accd5ad1 and 21d16f25 equal their HEAD blobs, git diff HEAD and git status --porcelain both empty, re-runs 7 and 14 passed.
  • Every exit code captured after a redirect, never through a pipe.

Generated by Claude Code

@os-musk
os-musk marked this pull request as ready for review September 7, 2026 19:14
@os-musk
os-musk enabled auto-merge September 7, 2026 19:14
@os-musk
os-musk added this pull request to the merge queue Sep 7, 2026
Merged via the queue into main with commit fc015bc Sep 7, 2026
35 checks passed
@os-musk
os-musk deleted the claude/issue-16225-protocol-harness-table-keyed-stub branch September 7, 2026 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] Eight protocol test harnesses store every table in one row map, so a sys_metadata_history row is served back as an active sys_metadata row

2 participants